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Abstract 

This  paper  shows  how  to  distribute  GPS-time  with  gs-accuracy  and  below  even 
in  Ethernet-based  distributed  systems.  Our  SynUTC- approach,1  is  based  upon  a 
simple  network  controller-level  hardware  for  timestamping  data  packets  as  they 
leave  and  arrive  at  a  node,  which  comes  in  two  flavors:  The  memory-based 
timestamping  method  exploited  by  our  Network  Time  Interface  (NTI)  M-Module 
timestamps  data  packets  as  the  network  controller  accesses  them  in  memory.  This 
technique  can  be  used  for  virtually  any  type  of  network  and  network  controllers. 

For  10  Mb/s  Ethernet,  for  example,  our  experimental  evaluation  revealed  a  time 
distribution  accuracy  down  to  the  gs-range.  Still,  memory-based  timestamping 
requires  network  controllers  that  cannot  store  entire  packets  on-chip,  and  the 
available  configuration  parameters  must  be  carefully  chosen  in  order  to  cope  with 
the  various  hidden  sources  of  timing  uncertainty.  To  escape  from  those  limi¬ 
tations,  we  recently  developed  a  novel  Mil-based  timestamping  method  that  can 
be  used  in  conjunction  with  almost  any  modem  10/100  Mb/s  Ethernet  chipset. 

The  timestamping  hardware  sits  at  the  standard  Mil-interface  between  network 
controller  and  transceiver  here,  and  will  allow  a  time  distribution  accuracy  well 
below  the  gs-range. 

1The  SvnUTC-project  received  support  from  the  Austrian  Science  Foundation  (FWF)  grant  P10244-OMA,  the 
OeNB  “Jubilaumsfonds-Projekt”  6454,  the  BMfWV  research  contract  Z1.601.577/2-IV/B/9/96,  the  Gesellschaft  fur 
Mikroelektronik  (GMe),  and  the  START  programme  Y41-MAT.  See  http://www.auto.tuvjien.ac.at/Projects/SynUTC/ 
for  further  information. 
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1  INTRODUCTION 


Next-generation  distributed  real-time  applications  will  certainly  be  equipped  with  ac¬ 
curately  synchronized  clocks  at  every  node.  In  fact,  synchronous  data  acquisition  and 
simultaneous  triggering  of  actuators  at  several  nodes  is  impossible  without  such  a  fea¬ 
ture.  Given  the  advances  in  GPS  technology,  it  is  no  longer  an  issue  to  supply  highly 
accurate  time  information  to  dedicated  computing  nodes.  However,  accurately  and  re¬ 
liably  disseminating  GPS-time  to  all  nodes  in  a  distributed  system  is  still  a  challenge. 

As  more  and  more  distributed  real-time  systems  are  now  being  built  atop  COTS- 
type  LANs,  the  most  desirable  solution  would  be  using  the  data  network  for  time 
distribution,  as  done  e.g.  in  NTP  [Mil91].  However,  [LWL84]  revealed  that  the  worst- 
case  synchronization  tightness  achieved  by  any  clock  synchronization  scheme  depends 
on  the  worst-case  uncertainty  (=  maximum  variability,  jitter)  e  in  the  end-to-end 
transmission  delay.  For  typical  LANs,  e  lies  in  the  ms-range,  which  makes  it  impossible 
to  use  a  simple  packet  data  exchange  to  disseminate  time  with  ^s-accuracy.  Additional 
techniques  are  required  for  this  purpose,  which,  however,  should  be  compatible  with 
existing  network  controller  technology  to  be  useful  in  practice. 

Part  of  our  research  project  SynUTC  [Sch94]  is  devoted  to  this  problem.  Among 
its  results  is  our  Network  Time  Interface  (NTI)  [SKM+00]  add-on  hardware,  which  is 
available  as  an  industry-standard  M-Module  [MUM96].  The  NTI  has  been  built  around 
our  custom  UTCSU-ASIC  [SSHL97],  which  contains  most  of  the  hardware  support 
required  for  interval- based  external  clock  synchronization  in  fault-tolerant  distributed 
systems:  A  high-resolution  state-  and  rate-adjustable  clock,  local  accuracy  intervals, 
interfaces  to  GPS  receivers,  and  various  timestamping  features. 

In  [SN99]  and  [SKM+00],  we  showed  that  a  worst-case  e  in  the  10  /js-range  can  be 
achieved  when  using  the  NTI  in  a  10  Mb/s  Ethernet-coupled  distributed  system  made 
up  of  Motorola  MVME-162  CPUs.  Although  e  can  be  brought  down  to  the  few  ^s-range 
in  more  suitable  system  architectures,  it  became  nevertheless  clear  that  memory-based 
timestamping  in  conjunction  with  existing  network  interfaces  is  limited  both  with 
respect  to  applicability  and  achievable  time  distribution  accuracy. 

In  this  paper,  we  present  and  analyze  a  complete  algorithm  for  distributing  GPS-time 
with  a  few  /xs-range  accuracy  in  our  NTI-based  testbed.  In  addition,  we  introduce  a 
novel  Mil-based  timestamping  method,  which  will  be  employed  in  a  second  generation 
10/100  Mb/s  Ethernet-NTI  that  is  currently  under  development.  The  presented  ma¬ 
terial  is  organized  as  follows:  After  a  brief  overview  of  the  NTI  and  its  pivotal  UTCSU 
in  Section  2,  we  introduce  our  evaluation  system’s  hardware  and  software  architecture 
in  Section  3.  Section  4  is  devoted  to  the  particular  time  distribution  algorithm  used. 
A  brief  survey  of  our  novel  Mil-based  timestamping  method  in  Section  5  and  some 
conclusions  in  Section  6  eventually  complete  the  paper. 

2  NTI  FEATURES  AND  ARCHITECTURE 

The  Network  Time  Interface  (NTI)  [SKM+00]  has  been  designed  for  high-accuracy  fault- 
tolerant  external  clock  synchronization  in  LAN-based2  distributed  systems.  Apart 
from  advanced  clock  synchronization  algorithms,  this  goal  primarily  requires  hardware 

2Note  that  the  approach  can  also  be  adopted  to  more  general  topologies  commonly  known  as  WANs-of-LANs, 
provided  that  all  gateway  nodes  are  also  equipped  with  the  NTI,  cf.  [Sch94]  and  [SS95]. 
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support  for  exact  timestamping  of  data  packets  and  a  sophisticated  rate  and  state 
adjustable  clock  at  each  node.  Figure  1  shows  the  basic  architecture  of  a  2-node 
system. 


Figure  1:  Basic  architecture  of  a  2-node  system 


Accordingly,  each  node  must  be  equipped  with  a  clock  device  (the  UTCSU,  see  below), 
a  CPU  responsible  for  executing  the  clock  synchronization  algorithm,  and  a  Communi¬ 
cations  Coprocessor  (COMCO)  providing  access  to  the  network  by  reading/writing  data 
packets  from/to  shared  memory.  At  least  one  node  must  also  be  connected  to  a  GPS 
timing  receiver. 

In  purely  software-based  clock  synchronization,  timestamping  of  Clock  Synchronization 
Packets  (CSPs)  at  the  sending  resp.  receiving  side  is  done  by  reading  the  clock  when 
assembling  the  CSP  for  transmission  resp.  in  the  COMCO’s  packet  reception  interrupt 
service  routine.  However,  as  explained  in  detail  in  [SKM+00],  this  implies  that  the 
transmission  delay  uncertainty  e  includes  both  the  network  channel  access  uncertainty 
and  the  reception  interrupt  latency,  which  can  be  quite  large.  To  get  rid  of  those,  a 
refinement  of  the  DMA-based  coupling  method  originally  proposed  in  [K087]  can  be 
used.  The  key  idea  is  to  insert  a  timestamp  on-the-fly  into  the  memory  holding  a  CSP 
in  a  way  that  minimizes  e,  as  outlined  in  Figure  2. 
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Figure  2:  Packet  timestamping 


Whenever  the  COMCO  fetches  data  from  the  transmit  buffer  holding  the  CSP,  it  has 
to  read  across  the  particular  offset  that  causes  a  special  decoding  logic  to  generate 
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the  trigger  signal  TTSXMT.  Upon  its  occurrence,  the  UTCSU  puts  a  transmit  timestamp 
(TxTS)  into  a  dedicated  sample  register,  which  is  transparently  mapped  into  a  certain 
portion  of  the  transmit  buffer  and,  hence,  automatically  inserted  into  the  outgoing 
packet.  By  the  same  token,  when  the  receiving  COMCO  writes  a  certain  offset  in 
the  receive  buffer,  the  trigger  signal  TTSRCV  is  generated  by  the  decoding  logic,  which 
causes  the  UTCSU  to  sample  the  receive  timestamp  (RxTS)  into  a  dedicated  register. 
This  timestamp  is  saved  subsequently  in  an  unused  portion  of  the  receive  buffer. 
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i  ms  approacn  worKS  ior  any  o<^iviv_/W  in  at  accesses  cor  uaua  ill  memory, 
chip-sets  are  available  for  a  wide  variety  of  networks.  It  must  be  stressed,  however,  that 
COMCOs  with  a  transmit-FIFO  that  can  held  an  ent ire  pachet  defeat  memory- based 
timestamping.  Moreover,  determining  e  for  a  particular  COMCO  usually  requires 
experimental  evaluation. 


The  NTI  provides  the  above  memory-based  timestamping  facilities  — as  well  as  all 


other  hardware  support  rec^mred  for  high- accuracy  cloch  synchronization” 
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height  (146  x  53  mm)  MA-Module3.  Figure  3  shows  the  major  components  on  board 
the  NTI,  which  can  be  accessed  from  any  COTS  CPU/COMCO  with  M A- interface  via 
ordinary  memory  and  memory-mapped  registers;  details  can  be  found  in  the  compre¬ 
hensive  user  manual  [MNS99]. 


Figure  3:  NTI  block  diagram 


All  required  decoding  and  glue  logic  of  the  NTI  is  assembled  in  a  single,  in-circuit  pro- 
pra.mmahlp  r.irm.nlp.T.  nrnnrnm.m.n.blp  Inair.  device  ( CPLD'l  desierned  usine  VHDL.  It  adaDtS 
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the  UTCSU  and  the  memory  to  the  MA-Module  interface,  forwards  interrupt  requests 
from  the  UTCSU  to  the  carrier-board,  generates  the  acknowledgment  signal  termi¬ 
nating  a  bus  cycle,  etc. 
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providing  the  special  functionality  for  COMCO  accesses,  as  outlined  before.  It  con¬ 
sists  of  up  to  four  64k  x  16  bit  SRAM  chips  and  supports  byte,  word,  and  longword 
read/ write  accesses. 


T1  U  T  TT'OOT  T_  A  OTC1  (  T  Tm  An  ■  7  T/iwvx  o  f^r\r\tr*Anrr\rrlaA  /^OO/'nKor]  in  rc;<mT.Q7l 
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and  [Loy96]  contains  most  of  the  dedicated  hardware  support  for  clock  synchroniza- 

3  M- Modules  [MUM96]  implement  an  open,  simple,  and  robust  mezzanine  bus  interface  primarily  designed  for 
VME  carrier  boards.  MA-Modules  are  enhanced  M-Modules,  providing  a  32  bit  data  bus  and  enhanced  addressing 
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tion.  Manufactured  as  an  ASIC  in  0.7  pm  CMOS  technology,  the  UTCSU  accommo¬ 
dates  about  80,000  gates  on  a  100  mm2  die  packed  into  a  208-pin  MQFP  case.  Figure  4 
gives  an  overview  of  the  major  functional  blocks. 


unit 

full  name 

ACU 

Accuracy  Unit 
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Application  Unit 

BIU 

Bus  Interface  Unit 
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Built-In  Test  Unit 
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GPS  Unit 

ITU 

Interrupt  Unit 

LTU 

Local  Time  Unit 

NTU 

Network  Time  Interface  Unit 

SNU 

Snapshot  Unit 

SSU 

Synchronization  Subnet  Unit 

Figure  4:  Interior  of  the  UTCSU 


The  centerpiece  of  the  UTCSU  is  a  local  clock  unit  (LTU)  utilizing  a  56  bit  NTP-time 
format.  Clock  time  can  be  read  atomically  as  a  32-bit  timestamp  with  a  resolution 
of  2-24  «  60  ns  and  a  32-bit  macrostamp  containing  the  remaining  24  most-significant 
bits  -)-  an  8  bit  checksum  protecting  the  entire  time  information.  The  LTU  employs 
an  unconventional  adder-based  clock  design,  which  uses  a  91-bit  adder  instead  of  a 
simple  counter  for  summing  up  the  elapsed  time  between  succeeding  oscillator  ticks. 
Owing  to  this,  the  UTCSU  can  be  paced  by  a  quartz  oscillator  of  arbitrary  frequency 
fosc  6  1...20  MHz;  alternatively,  an  external  frequency  source  like  the  10  MHz  output 
of  a  high-end  GPS  receiver  can  be  used.  Moreover,  the  local  clock  is  fine-grained 
rate  adjustable  in  steps  of  about  10  ns/s  and  supports  state  adjustment  via  continuous 
amortization  as  well  as  leap-second  corrections  in  hardware. 

To  support  interval-based  clock  synchronization  (see  Section  4),  the  UTCSU  contains 
two  more  adder-based  “clocks”  in  the  ACU  that  are  also  driven  by  the  oscillator  fre¬ 
quency  fosc.  They  are  responsible  for  holding  and  automatically  deteriorating  the  16-bit 
accuracies  ct~  and  a+ ,  thereby  maintaining  a  bound  on  the  local  clock’s  instantaneous 
deviation  w.r.t.  real  time. 

A  number  of  external  events,  supplied  to  the  UTCSU  via  buffered/opto-decoupled, 
polarity  programmable  input  lines,  can  be  time+accuracy-stamped  with  local  time  and 
accuracy  upon  the  appropriate  input  transition.  Optionally,  an  interrupt  can  be  raised 
on  such  an  event  as  well.  Three  different  functional  blocks  in  the  UTCSU  utilize  this 
feature:  First,  trigger  signals  generated  by  the  decoding  logic  at  CSP  transmission 
and  reception  sample  the  current  local  time+accuracy  into  dedicated  UTCSU  registers 
in  one  of  the  six  available  SSUs.  Second,  three  independent  GPUs  are  provided  for 
timestamping  the  one  pulse  per  second  (lpps)  signal  — indicating  the  exact  beginning  of  a 
second —  from  up  to  three  GPS  timing  receivers.  Finally,  nine  independent  application 
time-t- accuracy-stamping  inputs  are  provided  by  the  APU. 

Those  timestamping  features  are  complemented  by  several  48-bit  programmable  duty 
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timers:  Whenever  local  time  reaches  the  programmed  time  of  an  armed  duty  timer, 
an  internal  or  external  signal  changes  state  and  an  optional  interrupt  is  raised.  Duty 
timers  are  required  for  triggering  activity  of  the  clock  synchronization  algorithm,  for 
controlling  continuous  amortization,  inserting/deleting  leap  seconds,  and  generating 
application-related  events. 

All  application-related  I/O-pins  of  the  UTCSU.as  well  as  all  interfaces  to  GPS  receivers, 
are  routed  to  the  GPS  and  application  interface.  In  addition,  the  UTCSU’s  internal 
time  information  ( “NTPA-bus”  )  is  exported  for  future  expansion  modules.  Of  course, 
high-speed  opto-couplers  or  buffers  are  provided  for  all  inputs  to  ensure  a  decoupled 
and  reliable  interface. 


3  EVALUATION  SYSTEM 

Aiming  at  the  support  of  existing  technology,  it  was  only  natural  to  use  COTS  compo¬ 
nents  for  building  up  our  evaluation  testbed  as  well.  As  in  [SN99]  and  [SKM+00],  we 
employ4  VMEbus-based  nodes  consisting  of  a  Motorola  MVME-162  CPU  and  an  AcQ 
i6360  or  MEN  A203  passive  carrier-board  hosting  the  NTI  MA-Module.  All  nodes 
are  interconnected  via  the  CPU’s  Ethernet  port  using  10  Mb/s  thin-wire  technology. 
Figure  5  outlines  the  basic  hardware  architecture. 


Figure  5:  Evaluation  system  hardware  architecture 


A  dedicated  wire  — the  only  add-on  to  Figure  1  required  for  evaluation  purposes —  is 
provided  for  simultaneous  sampling  of  the  current  time  at  all  nodes.  It  connects  the 
HWSNAP-timestamp  inputs  of  all  NTIs  and  is  driven  by  an  output  of  a  dedicated  master 
node  (NTI  0). 

The  evaluation  system’s  software  consists  of  several  layers  shown  in  Figure  6.  Two 
layers  of  driver  software  written  in  C  make  the  NTI’s  features  available  to  the  clock 
synchronization  algorithm:  The  lower-level  NTI-Handler  [SM99]  is  responsible  for  ini¬ 
tialization/configuration  of  the  NTI  and  the  M-Module  carrier-board,  as  well  as  any 

4Actually,  VMEbus  equipment  from  another  research  project  was  re-used  for  this  purpose.  Whereas  the  resulting 
architecture  is  unlikely  to  be  chosen  in  practice,  it  nevertheless  constitutes  a  suitable  “worst  case  environment”  for 
evaluation. 
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Figure  6:  Architecture  of  a  pSOS*~m  node  using  the  NTI-  and  GPS-Driver 


low-level  interrupt  handling.  The  upper-level  driver  layer  consists  of  the  NTI-Driver 
and  the  GPS  Device-Driver. 

The  NTI-Driver  [RSS99]  allows  the  pSOS+m  kernel  and  other  software  components  to 
communicate  via  the  MVME-162’s  i82596  Ethernet-COMCO  with  or  without  data 
packet  timestamping.  It  actually  multiplexes  three  different  interfaces  to  the  COMCO: 

1.  Kernel  Interface  (KI):  pSOS+m  supports  multiprocessing  by  means  of  remote  objects 
(tasks,  queues,  semaphores,  etc.),  which  are  implemented  atop  of  remote  procedure 
calls  (RPCs).  To  keep  the  kernel  reasonably  independent  of  the  underlying  net¬ 
work,  a  user-supplied  KI  is  required  that  maps  a  simple  message-passing  interface 
to  the  particular  COMCO. 

2.  Network  Interface  (NI):  In  addition  to  kernel  services,  application  tasks  can  use 
TCP/IP  sockets  for  communication  with  remote  sites  if  the  additional  software 
component  pNA+  is  present.  Like  the  pSOS+m  kernel,  pNA+  is  kept  hardware- 
independent  by  means  of  a  user-supplied  NI,  which  is  similar  to  the  Kl.but  plugs 
into  a  different  message-passing  interface. 

3.  Clock  Interface  (Cl):  The  third  component  that  requires  network  services  is  the 
clock  synchronization  algorithm.  Again,  a  simple  message-passing  interface  Cl  is 
sufficient  here.  Note  that  it  is  the  only  one  that  relies  upon  the  timestamping 
feature  of  the  NTI. 


The  GPS  Device-Driver  [US99]  is  responsible  for  interfacing  one  or  more  GPS  timing 
receiver(s)  to  a  pSOS+rn-based  target  system  equipped  with  at  least  one  NTI.  As  shown 
in  Figure  5,  it  supports  GPS  timing  receivers  that  provide  an  RS232  serial  interface,  a 
1  pps  output  and  an  optional  digital  status  signal.  The  1  pps  signal  announces  when 
GPS-time/UTC  advances  to  the  next  second;  the  status  signal  indicates  a  valid/non- 
valid  output.  The  RS232  data,  whicbareusually  supplied  several  10  ms  after  the  1  pps, 
reveals  the  point-in-time  of  the  last  1  pps  transition,  usually  along  with  additional 
status  information.  The  GPS-Driver  assumes  that  the  RS232  interface  is  connected  to 
a  standard  pSOS+m  serial  channel,  whereas  the  1  pps  and  status  output  are  fed  to  one 
of  the  three  GPUs  of  the  NTI’s  UTCSU  (recall  Section  2i 
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The  GPS  Device-Driver  provides  the  clock  synchronization  with  an  easy  to  use  ap¬ 
plication  programmable  interface  (API)  through  which  a  GPS  receiver  can  be  controlled 
and  time  information  retrieved.  After  initialization  of  the  NTI  M-Module,  the  RS232 
interface  and  the  GPS  receiver,  the  driver  processes  the  1  pps  transitions  and  RS232 
messages  and  calls  a  user-supplied  callback  function  that  periodically  delivers  a  sub¬ 
microsecond-accurate  timestamp  of  the  last  1  pps  transition.  Note  carefully  that  this 
accuracy  can  only  be  achieved  due  to  the  hardware  timestamping  feature  of  the  GPU. 
Special  attention  is  paid  to  fault  recognition  (lost  or  wrong  1  pps  pulses,  lost  or  wrong 
RS232  messages)  and  error  handling,  since  wrong  1  pps  <->  RS232  message  assignments 
would  result  in  a  1  second-range  error. 

Viewed  at  the  level  of  application  tasks,  the  NTI/GPS-Driver  and  the  atop  run¬ 
ning  clock  synchronization  algorithm  simply  add  synchronized  clocks  to  a  standard 
pSOSTm/pNA+-environment.  Apart  from  the  created  CPU  and  network  load,  the  pro¬ 
cess  of  clock  synchronization  is  in  fact  totally  transparent  to  the  application.  Still, 
complex  distributed  timing  problems  can  now  be  solved  by  means  of  the  various  time- 
stamping  and  duty  timer  functionalities  of  the  UTCSU’s  APU. 

4  TIME  DISTRIBUTION  ALGORITHM 

During  the  last  few  years  of  working  on  the  project  SynUTC,  we  established  a  rea¬ 
sonably  complete  framework  for  fault -tolerant  interval-based  clock  synchronization 
[Sch94],  [SS97],  [Sch97c],  [Sch97b],  [Sch97a],  [SW99],  etc.  Real-time  t  {=  GPS-time 
or  UTC)  is  not  just  represented  by  an  ordinary  clock  Cx(t)  at  node  x  here,  but  rather 
by  an  interval  clock  Cx(t)  =  [C^tjia^t)]  made  up  of  the  UTCSU’s  clock  value  Cx(t)  and  its 
interval  of  accuracies  ax(t)  =  [— a~(t),a+(t)].  An  interval-based  clock  synchronization  al¬ 
gorithm  is  responsible  for  maintaining  any  Cx(t)  such  that  accurateness  w.r.t.  real-time, 
i.e.,  t  e  Cx(t),  as  well  as  mutual  precision ,  i.e.,  \Cx(t)  -  Cy(t)\  <  7rmax,  can  be  guaranteed. 

The  major  advantage  of  the  interval-based  approach  is  a  locally  available  on-line  bound 
on  the  local  clock’s  deviation  from  real-time:  Since  t  €  Cx(t)  just  means  t  €  [ Cx{t )  - 
af(t),Cx(t)  4-  a+(t)],  an  application  can  judge  whether  the  instantaneous  accuracy  is 
sufficient  for  a  certain  goal.  This  feature  is  particularly  interesting  for  multi-clustered 
applications,  since  accuracy  w.r.t.  real-time  implies  a  certain  precision  even  for  nodes 
that  do  not  participate  in  a  common-clock  synchronization  algorithm:  If  nodes  x , 
y  in  different  clusters  are  both  non-faulty,  inclusion  of  t  implies  that  their  intervals 
must  be  overlapping.  Hence,  clock  values  Cx(t),  Cy(t)  cannot  be  further  apart  than 
—  (QZ  (t)  ■+■  (t))  S  Uy  ( t )  —  Cx  ( t )  <  Qly  (t)  +0^(4). 

The  price  to  be  paid  for  this  accuracy  information,  however,  are  explicit  bounds  on 
certain  system  parameters  like  transmission  delay  uncertainty  e  —  and  this  is  where 
the  results  of  an  experimental  evaluation  like  [SN99]  and  [SKM+00]  come  into  play. 
Figure  7  shows  the  transmission  delay  uncertainty  e  observed  for  the  NTI  in  the  eval¬ 
uation  system  of  Figure  5.  Thanks  to  the  hardware  timestamping  capabilities  of  the 
NTI,  the  primary  transmission  delay  characteristics5  are  independent  of  the  network 
load.  It  is  apparent,  however,  that  excessively  long  transmission  delays  occur  now  and 
then.  This  forced  us  to  choose  a  conservative  bound  for  e  in  the  simple  algorithm  of 
[SKM+00],  which  thus  suffers  from  a  relatively  poor  worsUcase  accuracy. 

In  this  paper,  we  show  how  to  improve  accuracy  by  means  of  a  slightly  more  ad- 

sNote  that  the  second  peak  is  caused  by  packets  that  find  the  Ethernet  busy  upon  their  arrival. 
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Figure  7:  Histogram  of  transmission  delays  (A203  carrier  board) 


vanced  algorithm  using  multiple-source  time  distribution  combined  with  round-trip- 
based  transmission  delay  measurement:  Consider  a  system  consisting  of  m  >  1  P- 
nodes  pi ,  ...,pm  equipped  with  a  GPS  receiver,  and  one  or  more  S-nodes  that  are  to  be 
synchronized  with  GPS-time.  Assume  that: 

•  node  p’s  GPS  receiver  has  accuracy  ap, 

•  each  UTCSU  is  driven  by  an  oscillator  with  frequency  /0,  resulting  in  a  clock 

granularity  G  =  [-G,0]  with  G  =  and  a  rate  adjustment  uncertainty 

u  =  [— p,u]  with  u  =  l/f0  (see  [SS97,  Assum.  2]), 

•  any  node  x’s  UTCSU  is  set  up  properly  to  ensure  correct  deterioration  of  the 
local  interval  of  accuracies  ax  when  the  maximum  local  oscillator  drift  is  within 
px  =  [— px,px]  (see  below)  and  discrete  rate  adjustment  with  u  applies  (see  [SS97, 
Assum.  3]), 

•  the  transmission  delay  from  node  x  to  y  is  S'xy  e  \5xy  ±  exy],  with  unknown  — but 
constant  or  slowly  varying —  expectation  Sxy  =  5yx  +  dyx,  known  unsymmetry  dyx, 
and  known  maximum  uncertainty  exy  =  [— ej  ,e+  ], 

•  at  most  /  >  0  of  the  m  >  2/  +  1  P-nodes  may  deliver  wrong  time  information  to  an 
S-node  due  to  GPS  failures,  P-node  failures,  excessive  transmission  delays,  etc. 

Table  1  shows  the  particular  parameter  settings  for  our  evaluation  system  when  all 
nodes  are  equipped  with  an  A203  carrier  board. 


Name 

Value 

Meaning 

fo 

10  MHz 

UTCSU  oscillator  frequency 

U 

100  ns 

rate  adjustment  uncertainty 

G 

2-23  «  120  ns 

clock  granularity 

Px 

II 

T 
►— * 
o 

1 

»— * 
o 

1 

-J 

maximum  oscillator  drift 

Ctp 

ao  =  [—150ns,  150ns] 

accuracy  Motorola  VP-Oncore  GPS  receiver  [HS97] 

Exy 

e  =  [-3ps,  3psj 

transmission  delay  uncertainty  (see  below) 

dXy 

d=0 

average  unsymmetry 

A 

300  ms 

maximum  transmission  delay 

rs 

T  =  100  ms 

maximum  computation  time  S-nodes 

rP 

r0  =  10  ms 

maximum  computation  time  P-nodes 

Table  1:  Parameter  settings  for  our  evaluation  system  (identical  nodes  with  A203  carrier  board) 
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Figure  8:  Time  distribution  algorithm:  P-nodes 


Figure  9:  Time  distribution  algorithm:  S-nodes 


(L)  Lock  local  clock  to  GPS-time 

Upon  every  lpps  transition  occurring  at 
GPS-time  tt  with  Cp(ti)  =  Ti, 

1.  compute 

Ip  =  [tj  ±  ap]  +  u  +  G 

+TP  -Ti  +  ( Tp  -  Ti)Pp 

for  Tp  —  Ti  +  r„,  where  >  0  com¬ 
pensates  the  execution  time  of  this 
step, 

2.  SetupClockCorrection(Tp,  Ip). 

(R)  Receive  round-trip  CSP  from  S-nodes 

Upon  reception,  at  local  time  Tp,  of  a 
CSP  containing  the  local  time  Tsp  of 
transmission  at  S-node  s,  save  (s,T,p,Tps). 

(D)  Distribute  GPS-time  to  S-nodes 

Periodically,  when  Cp(t)  =  kP  in  round  k, 

1.  BroadcastCSP(S,  Vs  :  (s,T,p,Tps)) 

2.  k:=k  +  l. 


Basic  operations: 

•  BroadcastCSPfX,  data)  unreliably  sends  a 
CSP  containing  data  to  all  X-nodes,  ei¬ 
ther  by  hardware  broadcasting  or  mul¬ 
tiple  point-to-point  transmissions.  Each 
CSP  is  time+accuracystamped  with  the 
local  interval  clock  upon  actual  transmis¬ 
sion. 

•  SetupClockCorrection(time,  interval)  sets 
up  the  UTCSU’s  clock  correction  duty- 
timer  for  time  time  to  initiate  local  inter¬ 
val  clock  correction  towards  interval. 

•  CVfm(Ii,  provides  a  new  value  for 

the  local  interval  clock.  It  computes  a 
fault-tolerant  intersection  [Sch97b]  and 
[SS99]  of  its  input  intervals  that  tolerates 
at  most  /  faulty  or  non-existing  intervals, 
and  selects  a  suitable  point  (usually,  the 
midpoint)  within  that  interval  as  the  new 
clock  value. 


(S)  Send  round-trip  CSP  to  P-nodes 

Periodically,  when  Cs{t)  =  kP  —  A  in 
round  k,  BroadcastCSP(P,  0). 

A  >  0  must  allow  reception  at  any  P-node 
before  step  (D),  and  should  ensure  that 
there  is  no  step  (L)  during  this  time. 

(G)  Get  GPS-time  from  P-nodes 

Upon  reception,  at  local  time  Tf,  of  a 
CSP  from  node  p  containing  Cps  =  [Tps  ± 
aps]  and  ( s,Tsp,T£ )  in  round  k , 

1.  compute  AT,  =  Tf  -  Tsp ,  AT),  =  Tps  - 
Tp  and  the  current  transmission  de¬ 
lay  estimator 


dps 


XT,  A  Tp  dps  ^  ^ 


and  update  the  corresponding  run¬ 
ning  averages  AT,,  A Tp,  5ps. 

2.  compute 


Fs  =  Cps  +  u  +  G 

+  [<5P,  ±  £p«]  +  XT,p,  +  ATppp 
+TR  -  Tf  +  (Tr  -  Tf)ps,  (2) 

where  pp  =  [-p+,p~]  and  TR  =  kP  + 
A  +r,.  A  >  0  and  T,  >  0  must  allow 
message  reception  and  completion  of 
all  computations  before  step  (C).3. 

(C)  Compute  the  final  correction  interval 
At  time  C,(t)  =  kP  +  A  in  round  k , 

1.  initiate  local  clock  reading  delivering 
C,  =  [T,  ±  a,]  and  compute 

Is  —  Cs  +  TR  —  T,  +  (Tr  —  Ts)ps, 

2.  compute 

if  =  cvfm(Lln Is,..., I?nis),  (3) 

3.  if  if  ^  0  then 
SetupClockCorrection(TK,  if), 

4.  k  :=  k  +  1. 
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The  time  distribution  algorithm  shown  in  Figure  8  and  9  proceeds  in  rounds  of  duration 
P  seconds,  which  are  indexed  by  k  >  1  and  scheduled  according  to  each  node’s  local 
clock.  Apart  from  locking  a  P-node’s  UTCSU-clock  to  GPS-time  upon  every  lpps 
transition  in  step  (L),  each  round  ends  with  a  CSP  exchange  (S)  -»  (R)  and  (D)  ->•  (G) 
between  any  S-  and  P-node,  followed  by  a  resynchronization  of  the  S-node’s  interval 
clock  in  step  (C).  The  CSP  round-trip  is  used  for  both  measuring6  the  transmission 
delay  and  for  distributing  GPS-time. 

The  formulas  given  in  Figure  8  and  9  try  to  preserve  a  given  interval’s  accurateness 
by  elementary  techniques  developed  in  our  interval-based  framework;  see  [SS97]  for 
an  elaborate  discussion.  For  example,  to  ensure  accurateness  of  the  remote  clock 
estimation  eq.  (2),  the  accuracy  interval  Cps  received  from  node  p  must  be  enlarged  by 
eps  to  account  for  the  variable  transmission  delay  8'ps  e  [<5ps±eps]  ( delay  compensation );  the 
terms  involving  A Ts  and  A Tp  compensate  drift-induced  errors  in  the  computation  of  8ps. 
In  addition,  when  shifting  the  resulting  interval  from  Tf  to  resynchronization  time  TR,  a 
sufficient  enlargement  (deterioration)  of  the  shifted  interval  is  required  to  compensate 
Cs{ty s  drift  p’s  €  ps.  Recall  that  this  drift  compensation  is  performed  continuously  by  the 
UTCSU  in  hardware  as  well. 

The  remote  clock  estimations  If  are  intersected  with  the  S-node’s  own  information  Is 
and  eventually  fed  into  the  convergence  function  CV  in  eq.  (3).  This  step  ultimately 
allows  us  to  neglect  the  rare  excessive  transmission  delays  apparent  in  Figure  7  and 
choose  £ps  =  [— 3ps,3ps]  in  Table  1  instead:  Since  CV  tolerates  up  to  /  faulty  input 
arguments,  a  larger  /  (i.e.,  a  larger  m  >  2/+1)  allows  to  choose  a  more  risky  eps  without 
increasing  the  probability  of  an  S-node  failure.  The  intersection  with  I$  constitutes 
a  simple  clock  validation  technique  [Sch95],  which  is  effective  for  detecting  excessive 
faults  even  in  case  of  /  =  0.  Note  that  accurateness  of  CV’s  arguments  is  not  affected 
by  this  intersection  as  long  as  the  S-node  s  is  non-faulty. 

It  is  not  difficult  to  analyze  the  worst-case  accuracy  and  precision  of  our  algorithm. 
Since  we  established  in  [Sch97b]  and  [SS99]  that  the  result  of  a  suitable  convergence 
function  is  contained  in  the  intersection  of  its  m  -  2/-largest  non-faulty  input  intervals, 
it  follows  that  If  yields  node  s’s  interval  of  accuracies  as  immediately  after  resynchro¬ 
nization.  Plugging  in  the  simplified  parameters  from  Table  1  and  the  obvious  bounds 
ATS,  ATP  <  A,  we  obtain 


cts  C  a  —  (a0  +  2 u  +  G  +  Is  ■  p^  +  +  G  +  £  +  (3A  +  r)p^ 

—  cto  3 it  +  2G  -f-  £  +  (3A  -4-  F  +  ls)p. 

Moreover,  the  interval  of  accuracies  immediately  before  the  next  resyrichronization  is 
bounded  by 

a!  =  a  4-  Pp  +  u  +  G 

for  G  =  [0,G],  so  that  the  worst-case  precision  of  any  two  non-faulty  S-nodes  in  the 
system  evaluates  to  tt  =  |a'|.  Plugging  in  our  particular  parameter  values  finally  gives 
the  worst-case  accuracy  and  precision  shown  in  Table  2.  It  is  instructive  to  compare  this 
table  with  [SKM+00,  Table  3],  which  shows  the  corresponding  results  when  correctness 
is  secured  for  any  CSP  transmission  by  choosing  e  =  [-3ps,  14ps].  It  is  apparent  that  our 
more  advanced  algorithm  considerably  improves  the  achievable  worst-case  accuracy. 

6Note  that  we  omit  the  initial,  say,  10-20  round-trips  needed  for  setting  up  the  running  averages  for  simplicity. 
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50 

-3. 9. 3. 7 

[-9.oj8.9j 

17.9 

100 

-3.9,3. 7 

[-14.0,13.91 

27.9 

Table  2:  Worst-case  accuracy  and  precision 


Still,  the  roughly  Bernoullian  transmission  delay  distribution  in  Figure  7  suggests 
that  the  average-case  accuracy  should  be  considerably  better  than  the  above  worst- 
case  results  indicate.  To  verify  this  conjecture,  we  developed  a  pSOS+m  evaluation 
application  atop  of  the  general  software  architecture  of  Figure  6,  which  allowed  us  to 
monitor  and  statistically  evaluate  the  synchronization  accuracy  of  our  time  distribution 
algorithm:  A  master  node’s  duty  timer  is  used  to  generate  periodic  state  transitions 
at  the  HWSNAP-signal  fed  to  all  nodes  in  the  system  (cf.  Figure  5}  A  dedicated  task 
at  each  node  reads  the  timestamps  triggered  by  HWSNAP-transitions  and  sends  it  to 
a  dedicated  evaluation  task  at  the  master  node  for  processing.  The  interested  reader 
is  referred  to  [APSOO]  for  further  details  and  our  evaluation  results. 


5  NTI  ENHANCEMENTS 

Whereas  the  /xs-range  accuracy  achieved  by  our  memory-based  NTI  implementation 
should  be  sufficient  in  most  cases,  we  are  aware  of  some  more  demanding  applications. 
The  currently  most  challenging  one  is  on-line  fault  location  in  power  distribution  grids, 
i.e. ,  the  problem  of  finding  out  when  and,  in  particular,  where  a  fault  (cable  break, 
short  circuit,  partial  discharge  etc.)  occurred  in  a  — usually  buried —  power  cable.  One 
promising  solution  is  to  attach  detectors  at  the  power  trunks  in  each  power/transformer 
station,  which  timestamp  the  instant  when  the  transient  wave  emanating  from  a  fault 
location  arrives.  By  relating  the  timestamps  gathered  at  both  ends  of  a  cable,  it  is 
possible  to  determine  the  fault  location  on-line  within  a  few  10  meters.  Still,  as  the 
transient  waves  travel  about  200  meters/^xs,  a  precision  in  the  10  ns-range  is  mandatory 
for  this  application. 

However,  our  experimental  evaluation  [SN99]  and  [SKM+00]  revealed  that  memory- 
based  timestamping  cannot  be  improved  by  the  required  two  orders  of  magnitude: 
Large  on-chip  FIFOs  found  in  modern  COMCOs  severely  impair  e,  and  if  a  COMCO 
can  store  an  entire  packet  in  its  transmit  FIFO,  the  method  does  not  work  at  all. 
Another  drawback  of  our  current  NTI  is  the  required  CPU  intervention  for  moving 
receive  timestamps  into  the  CSP,  cf.  [HSS98]:  Since  receive  time+accuracystamps  are 
sampled  into  dedicated  UTCSU-registers,  they  must  be  moved  into  an  unused  portion 
of  the  received  packet  before  the  next  one  drops  in.  The  present  NTI  uses  a  high- 
priority  interrupt  service  routine  for  this  purpose,  which,  however,  generates  a  high 
interrupt  load  on  the  CPU  in  case  of  back-to-back  receptions.  Moreover,  as  M-Modules 
provide  a  single  interrupt  line  only,  any  UTCSU-interrupt  is  forced  to  be  a  high-priority 
one. 

In  order  to  circumvent  those  shortcomings,  we  recently  developed  a  novel  Media  In¬ 
dependent  Interface-based  timestamping  method  that  can  be  used  in  conjunction  with 
almost  any  modern  10/100  Mb/s  Ethernet  chipset.  Figure  10  outlines  the  basic  ar- 
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chitecture  of  our  next  generation  MII-NTI,  which  will  be  built  as  a  PCI  board.  Note 
that  supporting  Fast  Ethernet  technology  was  considered  mandatory,  since  it  is  widely 
used  in  office  automation  and  becomes  increasingly  popular  in  other  areas  as  well. 
For  example,  the  Fieldbus  Foundation  ( http://www.fieldbus.org )  and  the  Industrial  Automa¬ 
tion  Open  Network  Alliance  (IAONA,  http://www.iaona-eu.com/)  are  trying  to  establish  Fast 
Ethernet  technology  in  the  area  of  fieldbusses  and  automation,  respectively. 


External 
PCI  bus 


GPS  Receiver 
Application  Support 


Figure  10:  Basic  architecture  of  the  MII-NTI 


Mil-based  timestamping  exploits  the  fact  that  almost  all  Fast  Ethernet  controllers 
support  the  standard  IEEE  802.3-compliant  Media  Independent  Interface  (Mil)  to  physical 
layer  devices.  The  Timestamp  Logic  outlined  in  Section  2  is  placed  into  this  Mil  data 
path,  i.e. ,  timestamps  are  now  inserted  on  the  network  side  of  the  COMCO  rather  than 
on  its  CPU  side.  More  specifically,  controlled  by  a  state  machine  that  recognizes  CSPs 
by  means  of  a  special  type  field,  time+accuracystamps  are  latched  from  the  UTCSU’s 
multiplexed  NTPA-bus,  serialized  and  then  inserted  transparently  into  the  Mil  data 
stream.  Since  this  is  also  done  upon  reception,  no  CPU  intervention  is  required  for 
moving  the  receive  timestamps  anymore. 

The  MII-NTI  neither  incorporates  the  CPU  that  executes  the  clock  synchronization 
software  nor  the  Shared  Memory  that  holds  the  data  packets  (cf.  Figure  ]).  Appropriate 
host  components  must  be  present  instead,  which  can  access  the  board  via  its  PCI 
interface.  A  dedicated  Local  Bus  is  provided  on  board  the  NTI  to  ensure  that  both 
the  Fast  Ethernet  controller  and  the  UTCSU  can  be  accessed  by  the  CPU.  As  Fast 
Ethernet  chipsets  are  usually  equipped  with  an  integrated  PCI  interface,  we  chose 
PCI  as  our  local  bus  and  use  a  PCI  Target  chip  for  mediating  accesses  to  the  UTCSU. 
However,  PCI  boards  may  only  host  a  single  PCI  interface;  hence,  a  PCI-to-PCI  Bridge 
is  finally  used  for  interfacing  to  the  external  PCI  bus. 

All  remaining  interfaces  and  devices  are  implemented  in  a  similar  fashion  as  on  the 
current  NTI  M-Module. 

We  are  reasonably  convinced  that  the  MII-NTI  will  eventually  provide  a  transmission 
delay  uncertainty  e  in  the  few  10  ns-range.  Combined  with  a  faster  UTCSU  and  more 
advanced  clock  synchronization  algorithms,  we  should  therefore  be  able  to  achieve  a 
precision  and  accuracy  in  the  10  ns-range.  A  forthcoming  paper  will  elaborate  on  the 
detailed  description  and  experimental  evaluation  of  our  solution. 
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6  CONCLUSIONS 


We  presented  an  overview  of  our  SynUTC  time  distribution  approach,  which  facili¬ 
tates  fault-tolerant  distribution  of  GPS-time  even  in  Ethernet-based  distributed  sys¬ 
tems.  Rather  than  equipping  each  node  with  a  modular  GPS  receiver,  we  rely  upon 
a  small  network-controller-level  add-on  hardware  that  allows  exact  timestamping  of 
data  packets  as  they  leave  and  arrive  at  any  node.  With  memory-based  timestamping, 
as  employed  in  our  NTI  M-Module,  a  worst-case  synchronization  accuracy  down  to 
the  ps-range  can  be  achieved.  Compared  with  pure  software-based  timestamping,  this 
constitutes  an  improvement  of  three  orders  of  magnitude.  Our  novel  Mil-based  time- 
stamping  method  will  allow  to  improve  this  already  remarkable  result  even  further. 
Apart  from  achieving  a  synchronization  accuracy  in  the  10  ns-range,  it  can  also  be 
applied  to  100  Mb/s  Ethernet  and  network  controllers  with  large  FIFOs. 

The  NTI  in  conjunction  with  interval-based  clock  synchronization  algorithms  also  over¬ 
comes  both  fault-tolerance  limitations  and  practical  problems  inherent  in  any  “ded¬ 
icated  receiver” -solution:  GPS-receivers  do  deliver  wrong  lpps  pulses  now  and  then 
[HS97],  and  the  large  time-to-fix  may  cause  a  joining  delay  of  30  seconds  or  more  for 
newly  powered  up  nodes.  Last  but  not  least,  the  “forest”  of  antennas  required  for  a, 
say,  distributed  factory  automation  system  with  100  nodes  is  difficult  to  accommodate 
and  connect.  Our  approach  simultaneously  increases  the  fault-tolerance  degree  and 
decreases  the  number  of  GPS  receivers  required  in  the  system  —  without  additional 
(cabling)  costs,  by  using  the  existing  data  network  only.  The  only  price  to  be  paid  is 
some  moderate  hardware  support  and  decreased  precision/accuracy,  which  is  hopefully 
acceptable  for  most  applications. 


7  REFERENCES 
References 

[APS00]  Daniel  Albeseder,  Gerold  Pawelka,  and  Ulrich  Schmid,  Evaluation  of  NTI- 
based  GPS-time  distribution  over  Ethernet,  Technical  Report  183/1-97, 
Department  of  Automation,  TU  Vienna,  January  2000  (forthcoming). 

[HS97]  Dieter  Hochtl  and  Ulrich  Schmid,  Long-term  evaluation  of  GPS  timing 
receiver  failures.  In  Proceedings  of  the  29th  IEEE  Precise  Time  and  Time  Interval  Sys¬ 
tems  and  Application  Meeting  (PTTI’97),  pages  165—180,  Long  Beach,  California, 
December  1997. 

[HSS98]  Martin  Horauer,  Ulrich  Schmid,  and  Klaus  Schossmaier,  NTI:  A  Network 
Time  Interface  M-Module  for  high-accuracy  clock  synchronization.  In  Pro¬ 
ceedings  6th  International  Workshop  on  Parallel  and  Distributed  Real-Time  Systems  (WP- 
DRTS’98),  pages  1067-1076,  Orlando,  Florida,  March-April  1998. 

[K087]  Hermann  Kopetz  and  Wilhelm  Ochsenreiter,  Clock  synchronization  in  dis¬ 
tributed  real-time  systems,  IEEE  Transactions  on  Computers ,  C-36(8) :933— 939, 
1987. 

[Loy96]  Dietmar  Loy,  GPS-Linked  High  Accuracy  NTP  Time  Processor  for  Distributed  Fault- 
Tolerant  Real-Time  Systems.  Dissertation,  Technische  Universitat  Wien,  Faculty 
of  Electrical  Engineering,  April  1996. 


558 


[LWL84] 

[Mil91] 

[MNS99] 

[MUM96] 

[RSS99] 

[Sch94] 

[Sch95] 

[Sch97a] 

[Sch97b] 

[Sch97c] 

[SKM+OO] 

[SM99] 

[SN99] 


Jennifer  Lundelius- Welch  and  Nancy  A.  Lynch,  An  upper  and  lower  bound 
for  clock  synchronization,  Information  and  Control,  62:190—204,  1984. 

David  L.  Mills.  Internet  time  synchronization:  The  network  time  protocol, 
IEEE  Transactions  on  Communications ,  39(10):1482-1493,  October  1991. 

Thomas  Mandl,  Herbert  Nachtnebel,  and  Ulrich  Schmid,  Network  Time 
Interface  user  manual,  Technical  Report  183/1-87,  Department  of  Automa¬ 
tion,  Technische  Universitat  Wien,  January  1999  (in  German). 

MUMM,  ANSI/VITA  12-1996,  M-Module  Specification,  Manufacturers  and  Users  of 
M-Modules  e.V.,  1996. 

Gerda  Richter,  Michael  Schmidt,  and  Ulrich  Schmid,  i82596  NTI  Device- 
Driver  software  documentation.  Technical  Report  183/1-90,  Department  of 
Automation,  TU  Vienna,  February  1999. 

Ulrich  Schmid,  Synchronized  UTC  for  distributed  real-time  systems.  In 
Proceedings  19th  IFAC/IFIP  Workshop  on  Real-Time  Programming  (WRTP’94),  pages  101— 
107,  Lake  Reichenau,  Germany,  1994. 

Ulrich  Schmid,  Synchronized  Universal  Time  Coordinated  for  distributed 
real-time  systems,  Control  Engineering  Practice,  3(6):877— 884,  1995  (Reprint 
from  Proceedings  19th  IFAC/IFIP  Workshop  on  Real-Time  Programming 
(WRTP’94),  Lake  Reichenau/Germany,  1994,  pp.  101-107  ). 

Ulrich  Schmid,  Interval-based  clock  synchronization  with  optimal  preci¬ 
sion,  Technical  Report  183/1-78,  Department  of  Automation,  Technische 
Universitat  Wien,  July  1997  (submitted  to  Information  and  Computation). 

Ulrich  Schmid,  Orthogonal  accuracy  clock  synchronization,  Technical  Re¬ 
port  183/1-77,  Department  of  Automation,  Technische  Universitat  Wien, 
March  1997  (submitted  to  Chicago  Journal  of  Theoretical  Computer  Sci¬ 
ence). 

Klaus  Schossmaier,  An  interval-based  framework  for  clock  rate  synchro¬ 
nization  algorithms.  In  Proceedings  16th  ACM  Symposium  on  Principles  of  Distributed 
Computing ,  pages  169-178,  St.  Barbara,  USA,  August  1997. 

Ulrich  Schmid,  Johann  Klasek,  Thomas  Mandl,  Herbert  Nachtnebel,  Ger¬ 
hard  R.  Cadek,  and  Nikolaus  Kero,  A  Network  Time  Interface  M-Module 
for  distributing  GPS-time  over  LANs,  J.  Real-Time  Systems,  18(1),  2000  (to 
appear). 

Ulrich  Schmid  and  Thomas  Mandl,  Implementation  of  the  NTI  Device- 
Handler,  Technical  Report  183/1-86,  Department  of  Automation,  TU  Vi¬ 
enna,  January  1999. 

Ulrich  Schmid  and  Herbert  Nachtnebel,  Experimental  evaluation  of  high- 
accuracy  time  distribution  in  a  COTS-based  Ethernet  LAN.  In  Proceedings 
24th  IFAC/IFIP  Workshop  on  Real-Time  Programming  (WRTP’99),  pages  59—68,  SchloC 
Dagstuhl,  Germany,  May /June  1999. 


559 


[SS95]  Klaus  Schossmaier  and  Ulrich  Schmid,  UTCSU  functional  specification. 
Technical  Report  183/1-56,  Technische  Universitat  Wien,  Department  of 
Automation,  July  1995. 

[SS97]  Ulrich  Schmid  and  Klaus  Schossmaier,  Interval-based  clock  synchronization. 
J.  Real-Time  Systems ,  12(2):173-228,  March  1997. 

[SS99]  Ulrich  Schmid  and  Klaus  Schossmaier,  How  to  reconcile  fault-tolerant  inter¬ 
val  intersection  with  the  Lipschitz  condition.  Technical  Report  183/1-96, 
Department  of  Automation,  TU  Vienna,  September  1999  (submitted  to 
Distributed  Computing). 

[SSHL97]  Klaus  Schossmaier,  Ulrich  Schmid,  Martin  Horauer,  and  Dietmar  Loy,  Spec¬ 
ification  and  implementation  of  the  Universal  Time  Coordinated  Synchro¬ 
nization  Unit  (UTCSU),  J.  Real-Time  Systems,  12(3):295— 327,  May  1997. 

[SW99]  Klaus  Schossmaier  and  Bettina  Weiss,  An  algorithm  for  fault-tolerant  clock 
state  &  rate  synchronization.  In  Proceedings  18th  IEEE  Symposium  on  Reliable  Dis¬ 
tributed  Systems  (SRDS’99),  pages  36-47,  Lausanne,  Switzerland,  1999. 

[US99]  Martina  Umlauft  and  Ulrich  Schmid,  GPS  Device-Driver  software  docu¬ 
mentation,  Technical  Report  183/1-97,  Department  of  Automation,  TU 
Vienna,  October  1999. 


560 


